WORLD INTELLECTUAL _ __ r 



PROPERTY 



PCT 

INTERNATIONAL APPUCATION PUBUSHED UNDER 




ORGANIZATION 

THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 7 : 
H04M3/56 



Al 



(43) 



International Publication Number: 
Internationa] Publication Date: 



WO 00A9693 

6 April 2000 (06.04.00) 



(21) Internationa] Application Number: PCT/CA99/00875 

(22) International FQing Date: 24 September 1999 (24.09.99) 



(30) Priority Data: 
60/101,857 
2,264,407 



25 September 1998 (25.09.98) US 
4 March 1999 (04.03.99) CA 



(71) Applicant (for all designated States except US): WIRELESS 

SYSTEM TECHNOLOGIES, INC. [CA/CA]; Suite 601, 
205 Richmond Street West, Toronto, Ontario M5V 1V3 
(CA). 

(72) Inventors; and 

(75) Inventors/Applicants (for US onfy): SNELGROVE, William, 
Martin [CA/CA]; Apartment 603, 90 Sherboume Street, 
Toronto, Ontario M5A 2R1 (CA). STUMM, Michael 
[CA/CA]; 3 Belvale Avenue, Toronto, Ontario M8X 2A6 
(CA). DE SIMONB, Mauricio [CA/CA]; Apartment 702, 
10 Queen's Quay West, Toronto, Ontario M5J 2R9 (CA). 

(74) Agents: O'NEILL, Gary et al.; Gowling, Strathy & Henderson, 
Suite 2600, 160 Elgin Street, Ottawa, Ontario KIP 1C3 
(CA). 



(pi) Designated States: AB, AL, AM, AT, AU, AZ, BA, BB, BG, 
BR, BY, CA, CH, CN, CR, CU, CZ, DE, DK, DM, EE, 
ES, FI, GB, GD, GB, GH, GM, HR, HU, ID, IL, IN, IS, JP, 
KB, KG, KP, KR, KZ, LC, LK, LR, LS, LT, LU. LV, MD, 
MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, 
SE, SG, SI, SK. SU TJ ( TM, TR, TT, TZ, Urt, UG, US, 
UZ, VN, YU, ZA, ZW, ARPO patent (GH, GM, KE, LS, 
MW, SD, SL, SZ, TZ, UG, ZW), Eurasian patent (AM, AZ, 
BY, KG, KZ, MD, RU, TJ, TM), European patent (AT, BE, 
CH, CY, DE, DK, ES, FI, PR, GB, GR, IB, IT, LU, MC, 
NL, PT, SB), OAPI patent (BP, BJ, CP, CG, CI, CM, GA, 
GN, GW, ML, MR, NB, SN, TD, TG). 



Published 

With International search report. 

Before the expiration of the time limit for amending the 
claims and to be republished in the event of the receipt of 
amendments. 



(54) Title: METHOD AND SYSTEM OF TELECONFERENCING 




AVAILABLE COPY 



(57) Abstract 



aid 



The present invention relates generally to telecommunications, 
Existing telephony systems suffer from a number of problems including 
on fixed hardware which results in long time to bring new products to 
personal computers, without the network performing any processing of 
The invention provides for an intelligent server which executes a separat 
available to that user. This allows for an open and flexible teleconference 



market 



more specifically, to a system and method of teleconferencing, 
system complexity, limited access and implementation of services 
Internet applications implement teleconferencing on end user 
lata streams or guaranteeing quality of service in the transmission, 
mixer for each user, where the mixer is dedi c ated to the resources 
system which can operate over multiple networks. 
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Method and System of Teleconferencing 

Field of Invention 

The present invention relates generally to telecommunications, and more 
5 specifically, to a system and method of tete< tonfertncing. 



applies tions 



Background of the Invention 

Teleconferencing systems allow 
they were in the same room. In spite of the 

1 0 these systems, they are commonly used for 
resulting reductions in travel time and cost, 
not generally be rationalized in other 
so teleconferencing is not common in these 
Traditional teleconferencing systems 

1 5 monophonic speaker arrangements at each 
teleconference, and the methodology was to 
participants, blocking the remainder of the 
evolving and systems are now available 
signals of the participants and stereo sound 

20 present even greater demands on the carrie(* 
and lower latency, which results in even 
explains the limited availability and use of 
Generally, each teleconferencing 
specific communication network. Presently, 

25 dominant: the public switched telephone 
These systems are typically composed of 
personal computers, an access network sucfi 
link, and a backbone network such as the 
or the intercity data networks. Although the 

30 greatly, the backbone networks must handle 
operate reliably and efficiently. Therefore, 
focussed on the provision of single sen/ices 
incentive for telephone companies to offer 
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peo sle at different locations to converse as if 
currently high cost and complexity of 
business applications, because of the 
However, the cost and complexity can 

such as academia and private use, 
[areas. 

consisted of single microphone and 
physical location participating in the 
broadcast the loudest voice to all other 
voices. However, the art has been 
offer such added features as video 
Generally though, these new features 
networks in terms of higher bandwidth 
cost and complexity. This largely 
advanced teleconferencing systems, 
is designed to be used with a 
two communication networks are 

for voice, and the Internet for data, 
equipment such as telephones or 
as a telephony local loop or a radio 

switched telephone network (PSTN) 
leeds of users at the terminals vary 
highly standardized loads in order to 
traditional communication networks 

;ather than differentiation. There is no 
varied features or to serve small niche • 



(which 



higher 
stcha 
system 



network) 



teiminah 



pt blic i 
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markets as the revenues would not offset the 
implementing these additional products. 

In voice telephony, services are implemented 
programs running on centralized switches which 
databases. The local databases specify whidji 
the switch software interprets these feature lis 



substantial cost of developing and 



behaviour, and the switch software also 
Common Channel Signalling System No. 7 



sei vices 



tie 



for telecommunications that defines the 
1 0 elements in the public switched telephone 
messages for basic call setup, management, 
intelligent or database services such as focal 
(800/888) services and call forwarding. - 
In PSTN, a user only has access to 
1 5 carrier, which in turn may only function within 
Therefore, users can only access the switches 
not be added by outside parties. 

Telephony features, such as teleconferencing 
adding code to the programs running the switches 
20 to the telephony network. The features available 
local databases accessed by the switch software 
may involve changing these databases together 
them, and may also involve purchasing and installing 
network. 

25 This limits the speed with which new 

hardware and software must be designed, tested 
inflexible assignment of tasks also makes it imbossible 
different types of hardware, for example to use 
with an overload of voice-conferencing or to provision 

30 feature. 

A traditional PSTN teleconferencing system 
bidirectional audio communication link with 
Typically, the system includes a microphone a 
signal from that location and a transport network 



by having large computer 
interrogate local and distributed 
features are enabled on a given line, 
:s and Implements the switch 
interrogates the distributed databases via 

(S 37) queries. SS7 is a global standard 
procedures and protocol by which network 
network (PSTN) intercommunicate control 
tear down, as well as for special 
dumber portability (LNP), toll-free 
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provided by the local exchange 
bounds of the SS7 protocol, 
in a limited way, and new features can 



, may only be implemented by 
or by adding specialized hardware 
to particular users are defined in the 
/and adding a new type of feature 
with the switch software that uses 
new types of hardware in the 



fe atures 



can be introduced since new 
, manufactured and deployed. The 
to share loads between 
idle tone-decoding hardware to help 
a new teleconferencing 



provides each user with a 
n of a plurality of remote transceivers, 
each location for producing an audio t 
such as the public switched 



10 



30 
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telephone network (PSTN) which deliveries each voice signal to a conference bridge 
This conference bridge mixes the voice signals and returns them to audio amplifiers 
and speakers at each location. 

The conference bridge is implemented as a new hardware component 
connected to the switch providing the servi:e. Adding a new feature such as Dolby 
noise reduction or bass boosting requires £ physical change to the hardware and/or - 
software in every switch that offers the service. 

Changes to existing telecommunication networks are therefore very 
complicated to make. There is a rigid mod ?l and hardware structure is difficult to 
extend, so existing telephone companies a *e forced to focus on broad services. 
When they do develop new products they inevitably take a long time to bring to 
market and are expensive to implement. 

Telecommunications systems need [to process the data flowing through in 
complex ways, often with processing occur ing on computer systems separated both 



communications paths are simultaneously 



active, and the processing applied to the vs rious flows of data changes frequently 
and in a wide variety of ways. The software needed to control these computer 
systems is generally large, complex and dif Icult to change. 

The complexity of present telecomnr unications systems software, and the 

20 extensive interactions between its software components, makes the development of 
new features very difficult. As well, teleconr munication services have traditionally 
been provided by large monopolies who em ployed proprietary equipment that onfy 
they had access to. Large telephone comp anies hesitate to allow open access to the 
control of their switches and servers due to the risk of failures and the resulting 

25 damages that would occur; therefore, only v ery limited access is allowed. 



companiesis therefore limited to a 
reduces the talent pool available and 



Software development for telephone 
"closed" group of trusted developers, which i 
shuts out developers with new ideas for niche markets. 

In summary, problems with the PSTN include: 

1 . system complexity results in long tirr e to bring new products to market; 

2. cost of services results in focus on few specific services rather than diversity 
and niche markets; 
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3. existing services are provided by dec icated hardware and software which are 
inflexible and must be physically, anc often manually, modified to offer new 
services or features; and 

4. only proprietary access to switches and their software code is allowed. 

5 The implementation of software applications In an Internet environment is 

generally done by the software running at th< 5 endpoints, and the IP (Internet 
Protocol) network is treated merely as a con luit for transfer of data packets between 
the two points. The routers in the IP networt; merely index internal routing tables 
using the address of data packets so that theiy know how to forward them, and do not 

1 0 generate data for either of the endpoints, or i eact to instructions from either of the 
endpoints. The Internet itself may be envisioned as a series of routers 
interconnected by an Internet backbone network designed for high-speed transport of 
large amounts of data. Users may access the Internet using personal computers in a 
number of manners including modems connected to the Public Switched Telephone 

15 Network (PSTN), or set top boxes connectec to existing telephone or television cable 
networks. 

Communications over the Internet can be administered using various 
protocols, over a variety of physical transfer media. A protocol is a set of conventions 
or rules that govern transfer of data between hardware devices. The simplest 

20 protocols define only a hardware configuration while more complex protocols define 
data formats, error detection and correction techniques and software structures. 

The key advantages of a protocol like IP are that it allows a large network to 
function efficiently and that it offers a standardized means by which applications 
software can use that network. The main disadvantages are: 

25 1 . that it does not allow processing to be | 
2. that it does not allow quality of service 1 

For example, the Internet generally w II not let a user run an applet on a node 
or server. This limitation is due to the architecture of the Internet which is based on 
the international OSI (Open Systems Interconnection) standard. The OSI standard 

30 describes communication systems using a seven layer model, each layer being 

operable to perform certain functions. Although OSI is not always strictly adhered to 
in terms of keeping related functions together in a well-defined layer, most 
telecommunication products make an attemp: to place themselves in relation to the 
OSI model. The OSI standard is not likely to change dramatically, nor is the 



performed on data streams; and 
to be specified. 
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Internet's use of the standard, so the Internet will not likely become an active 
component in the provision of telecommunication services. 

More importantly, the Internet does n at allow quality of service to be specified. 
Internet communications generally rely on ths transport of data packets over various 
heterogenous networks, so even though certain links may have predictable data 
rates, for example, a privately owned T1 line total end to end transfer rate is still not 
predictable or dependable. 

Some protocols such as resource reservation protocol (RSVP) set tags and 
priorities which can influence the routers on itn Internet path a little, but not a great 
deal. The RSVP is an extension to IP that permits specification of quality of service 
at a technical level, in terms of parameters such as data rates and latencies. It has 
had limited acceptance due to the complexity it adds to backbone networks and the 
need for their switching hardware to be upda ed. As well, little is accomplished 
unless all switches in the end to end connect on are responsive to the protocol, which 
is not generally the case. 

Therefore, typical software applicatior s operating over the Internet, such as 
teleconferencing, look at the Internet as simp y a transport network without any 
processing capability and all functionality is p aced at the participants locations. 
Implementations of teleconferencing over Internet, for example, have software at 
each user's personal computer (PC) that acts as the interface with the user, 
converting voice to data packets for IP transmission to each of the other participants 
in the teleconference. Accordingly, the user's PC also receives streams of voice data 
from each of the other participants In the teleconference and plays them through a 
sound card. 

This implementation suffers from severe scalability problems. For example, if 
there are ten participants in a teleconference, then each participant would require 
sufficient bandwidth to download nine simulta leous voice data streams from the 
other participants, in real time. As the bandw dth to each user would increase linearly 
with the number of participants, and the load i >n the network increase with the square 
of the number of participants, there would be an immense load on the network 
resources. Clearly, this is impractical for teleconferences with a large number of 
parties or services which themselves require high bandwidth such as video or high 
quality voice. Even if the bandwidth could be obtained, there is no way to ensure that 
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capital 



it is consistently available, as there is no 
Internet applications. 

As noted above, typically, each exist 
to operate over a particular network and is n 
5 varied networks now available. These networks 
network (PSTN), Internet, cellular telephone 
area networks (LANs) and wide area networks 
are a variety of media including optical fibre, 
connections, which execute communications 

1 0 digital format using a variety of different protocols 
been widely implemented, at considerable 
be quickly abandoned and a new, standard, 
constructed. Therefore, there is a need for c 
implementing teleconferencing over a mixed 

15 networks. 

Asynchronous Transfer Mode (ATM) 
protocols for addressing packets of data and 
typically been deployed in the core of backbone 
speeds at which ATM equipment operates 

20 accessible and because of the complexity of 
these mechanisms have not been used by 
Besides the IP and ATM networks 
networks such as Frame Relay and Ethernet, 
carry data, for example using trellis coding which 

25 signal commonly; which is commonly used by 
are also evolving of each major type of netwqrk, 
between implementations of these networks 
complexity induced by this variety makes it difficult 
to exploit all the networks available, and to 

30 Feature development is already difficult 

teleconferencing over voice networks. As 
messaging, shared files and whiteboards are 
products, and new applications such as distance 
Internet gaming, develop, the problem is becoming 
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wa: f to specify quality of service (QoS) in 

ng teleconferencing system is designed 
Dt capable of cooperating with the many 

include public switched telephone 
systems, satellite communications, local 
(WANs). Within these networks there 
wireless or hardwired electrical 
over these networks in analogue or 
. Many of these networks have 
cost, so it is unlikely that they will 
world-wide telecommunications network 
system which is capable of 
combination of communications 

networks, for example, use standard 
setting up connections, and have 
networks because of the high 
Because ATM routers are not directly 
their mechanisms for describing QoS, 
applications software, 
mentioned above, there are other data 
As well, the PSTN may also be used to 
maps digital data onto an analogue 
Personal Computer modems. Variants 
, and engineering differences 
rfesult in different performance. The 
for users and application software 
any to its fullest extent 
for the simple application of 
media such as videophone, typed 
mixed with traditional teleconferencing 
learning, Internet Relay Chat and 
even more severe. This problerfi 
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will grow even greater as expectations develop for features from one domain to be 



mapped into another, as when customers 



expect a feature similar to call-waiting to 



apply in videoconferencing or Internet ganr ing 



Furthermore, even for a single app 



needs, for example, requiring different degrees or forms of encryption. Therefore, 
there is a need for a system which can allow many cases and features without 
becoming complex, slow to develop and slow in operation. 

There is therefore a need for a method and system of teleconferencing that 
may be implemented over mixed telecommunications networks, and addresses the 
complexity of such existing networks to provide an open, scalable and flexible 
architecture. 
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ication, different users may have different 



input 



Summary of the Invention 

It is therefore an object of the invention 
teleconferencing which addresses the problems 

One aspect of the invention is broadly 
teleconferencing comprising: three or moro 
input and an audio output; a telecommunications 
terminals and operable to transport data tc 
modular mixing software for each respective 
telecommunications network, and operablf 
the audio outputs of the others of the user 
audio signals into a signal for the audio 
correlates to the needs of the respective u|ser 

Another aspect of the invention is 
comprising: means for interconnecting us^r 
from the user terminals; means for executing 
each respective user terminal, the separata 
means for receiving separate audio signals 
user terminals; and means for combining t 
the audio input of the respective user termpnal 
respective user terminal. 

An additional aspect of the inventic 
teleconferencing comprising the steps of: 



to provide a method and system of 
described above, at least in part, 
defined as a system for 
user terminals, each having an audio 

network interconnecting the user 
and from the user terminals; separate 

user terminal, executing on the 
: to receive separate audio signals from 
terminals; and to combine the separate 
of the respective user terminal which 
terminal. 

(jefined as: a server for teleconferencing 
terminals and transporting data to and 
separate modular mixing software for 
modular mixing software including: 
from the audio outputs of the others of the 
he separate audio signals into a signal for 
which correlates to the needs of the 



n is defined as: a method of 
ifeceiving, at a separate modular mixer 
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user terminals and executing on a 



representing a respective one of three or more 
telecommunications network, separate audio signals from audio outputs of the others 
of the user terminals; and combining the separate audio signals into a signal for an 
audio input of the respective user terminal whip correlates to the needs of the 
respective user terminal. 

A further aspect of the invention is defirfied as: a computer data signal 
embodied in a carrier wave, said computer data signal comprising a set of machine 
executable code being executable by a computer to perform the steps of: receiving, 
at a separate modular mixer representing a respective one of three or more user 
terminals and executing on a telecommunications network, separate audio signals 
from audio outputs of the others of the user terminals; and combining the separate 
audio signals into a signal for an audio input of the respective user terminal which 
correlates to the needs of the respective user 1 erminal. 

A still further aspect of the invention is iefined as: a computer readable 
storage medium storing a set of machine executable code, the set of machine 
executable code being executable by a computer server to perform the steps of: 
receiving, at a separate modular mixer representing a respective one of three or 
more user terminals and executing on a telecommunications network, separate audio 
signals from audio outputs of the others of the 
separate audio signals into a signal for an aud < 



which correlates to the needs of the respective user terminal 



will become more apparent from the 
to the appended drawings in which: 
system in a broad manner 



Brief Description of the Drawings 

These and other features of the inventibn 
following description in which reference is made 
Figure 1 presents a physical layout of a teleconferencing 

of the invention; 

Figure 2 presents an exemplary physical layout of a teleconference system in a 

preferred embodiment of the invention; 
Figure 3 presents a block diagram of exemplary 

preferred embodiment of the invention; 
Figure 4 presents a block diagram of an exerri plary 

a preferred embodiment of the invention 



user terminals; and combining the 
o input of the respective user terminal 



signal processing software in a 
and 

operating system architecture in 
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ffomi 



Detailed Description of Preferred Embod 

A system which addresses the objects 
physical layout in Figure 1. This figure 
between three or more user terminals 12, 
5 audio output. The phrase terminal" is used 
. suitable manner of user audio input and 
telephones, and personal computers with 
The audio input and output refer to the conn 
telecommunications network, and not to the 
10 terminal. 

A telecommunications network 18 
16 and has the necessary functionality to 
telecommunications network 18 also executes 
each respective user terminal 12, 14 , 16. 
1 5 operable to receive separate audio signals 
to combine those separate audio signals intc 
transported to the audio input of the respective 
correlates with the needs of that respective 
That is, if there were three participants 
20 three mixers, a first mixer for participant A, 
participants B and C, a second mixer for pari 
third mixer for participant C, which mixes A a 

The use of individual mixers 20, 22, 
network 1 8 addresses a number of the 
25 Firstly, having individual mixers 20, 

to its own user and to be tailored to the limits 
resources of the network and network 
example, the mixer of a user having a high 
quality stereo to its user, with balanced mixinjg 
30 user connection via an analogue, monophon 
strongest voice to its user, blocking voice 
reducing noise. 

Having a single mixer for all participants 
require an immense piece of software code 



24 



i problems 



21 



connexion 
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ments of the Invention 
outlined above, is presented as a 
presjents a system 10 for teleconferencing 
, 16, each having an audio input and an 
generally in the art to describe any 
device including telephones, cellular " 
microphones and speakers or headsets, 
actions between the terminal and the 
audio interface between the user and the 

intsrconnects these user terminals 12, 14 , 
transport data packets between them. The 
separate modular mixing software for 
separate mixers 20, 22, 24 are 
each of the other user terminals and 
one signal. This one mixed signal is 
user terminal, in a manner that 
User terminal. 

in the teleconference, there would be 
Which mixes the audio output signals of 
cipant B, which mixes A and C, and a 
nd B. 

executing on the telecommunications 
noted above. 
:, 24 allows each mixer to be dedicated 
ions of the user's resources and the 
that services that user. For 
bandwidth connection may provide digital 
of all participant's voices. Another 
c, PSTN connection may send only the 
signals from other participants and thereby 

, as taught in the PSTN art, would . 
all of the variations in user 
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requirements were to be handled in a single 
software code would be unmanageably large, 
have simply not offered such diverse services 
Similarly, the use of mixers 20, 22, 
5 network 18 offers a substantial improvemen 
teleconferencing as well. Typically, Internet 
all participants, so each terminal receives 
participants. This places a tremendous demand 
connection to each user and a tremendous 

1 0 invention requires only the number of audio 
audio output, to be sent to the user. That is 
only one channel is required, and if stereo is 
sound, surround sound, central bass and 
corresponding numbers of audio channels. 

1 5 required to each user and the loading on the 
Thirdly, it is also significant that the 
a modular manner. As will be described in 
software components of the invention are 
small modules designed to handle very spedific 

20 than those like the existing PSTN. The mom 
addresses, the easier it is to design that module 
of the software system. This is fundamental 
flexible and open. 

Other advantages of the invention wi 

25 description of the preferred embodiment whifch 
example. 

Figure 2 presents an exemplary physical 
preferred embodiment of the invention, havirjg 
have direct access to an active network, whi 
30 term "active network" refers to a network tha 

software and other related software components 

Participant A has a personal computer 
network 28 via a wireless connection 30. Th 
serves Participant A is called a NetPort 32 



PCT/CA99/00875 



piece of software. As this piece of 
complex and slow, existing systems 

2(4 executing on the telecommunications 
over existing Internet based 
methods broadcast all voice streams to " 
to (N - 1) streams where there are N 

on the bandwidth of the final 
oad on the network, in contrast, the 
channels that the user requires at his 
if the user desires monophonic output, 
desired, two channels. Quadraphonic 
er audio arrangements would require 
This greatly reduces the bandwidth 
network. 

nfiixers of the invention are implemented in 
£ reater detail hereinafter, all of the 
implemented in small modules. Having 
tasks results in a far simpler system 
defined the task that a module 

and later, to integrate it into the rest 
to the provision of a system that is 

become more apparent from the 
will be presented in terms of an 



layout of a teleconference in the 
four participants. Two participants 
e two are connected to the PSTN. The 
is operable to execute the mixer 

described hereinafter. 
(PC) 26 connected to a first active 
e entity on the active network 28 which 
he specific role of the NetPort 32 within 
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active network 28. Participant B also 
C 44 through a streaming-audio 
telephone 46 for its boom microphone. 



the active network 28 will be described in greater detail hereinafter. The PC 26 is 
running a stereo-enabled Web browser with a RealAudio piugfn that implements 
streaming audio and is output at speaker a I. The PC 26 also has a simple 
microphone plugin that passes samples fro n the on-board microphone 36 back to an 
IP address. Participant A also has a WebCam 38 connected to her PC 26. 

Participant B is connected directly to a second active network 40, but in a 
location geographically remote from the firs : 
listens from a speaker 42 connected to his 
application, but is talking through a wireless 

Participant B is plugged into a second NetPort 48 through a telephone jack and 
hardwired connection 50. 

Participant C is connected to the PCjTS 52 {plain old telephone system) via a 
plain black rotary-dial telephone 54. 

Participant D has two speaker phones 54, 56 fed by two separate POTS lines. 
He also has an Internet connection via a PC 60 which runs a Web browser, but his 
Internet Service Provider (ISP) does not provide good enough quality of service 
(QoS) for voice, so he just uses it for the gniphic user interface (GUI). 

A GUI is piece of software that prese nts data to users in a graphical manner, 
allowing for easy interpretation and modification. It is preferred that the invention be 
implemented in such a manner, where poss I 

browser, and communicates with call proceeising applications running on the active 
network by means of sockets. Invoking it involves typing a URL (uniform resource 
locator such as "coolPhones.com"), after which it sits in a window waiting for an 
incoming call or a user input event to place n call. Inputs can be made via a mouse, 
keyboard, trackball, touchscreen, joystick or < 



other similar manner. The GUI is strictly 
an interface, though, since it is unacceptable for example, to have voice-mail fail 
when the PC is not active. Therefore, the real call processing decisions are made on 
the active network side. 

This exemplary system also includes an Internet network 62, which is 
30 connected to the PSTN using, for example, H.323 and SIP (Session Initiation 

Protocol) connections. These connections are known in the art, as are others. The 
Internet network 62 is also shown to be connected to both active networks 28, 40, but 
many other system topographies are also pqssible. The invention is not limited by 
any particular topography. 
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progress, 



output 
currently speaking. 



full 



While the teleconference is in 
the left, Participant C in the middle and Part 
A is mixing the other participant's monophorjiic 
stereo spectrum. The use of the stereo 
5 aids in identifying which participant is 
noise levels to be tolerable to participants 
effect acknowledges that people are able to 
an environment where there is considerable 
have a means for identifying and focussing 

1 0 The use of stereo sound has been shown to 
Stereo sound can be synthesized in 
implementation, amplifier gain can be varied 
example: one participant may be played at 
all on the right, a second may be played witt 

1 5 none on the left, while a participant may be 
More complex implementation of stereo may), 
audio signal before playing it on one of the 
the sound takes to travel to the farther of the 
environment Such methods are generally 

20 Accordingly, Participant A's GUI 

B's picture on the left, Participant C's numbejr 
ID name on the right. In each case, the GUI 
information that it has available for each 
URL or telephone number. As well, 

25 which identifies personal information about 
As well, via the GUI, Participant A caih 
of the three participants on her screen to ens ble 
to see her through the WebCam. Participant 
the volume level at which they speak to her, 

30 adjust their stereo imaging. Alternatively, the 
governed by their physical location on the 
is away from an icon representing the user, 
preferred feature of the GUI is that when 
brighter then gradually fades again with inactivity 



sere an 



partcipant. 



screen- pop 



tthe 



GUI 



tie 
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, Participant A hears Participant B on 
cipant D on the right, because Participant 
voice streams into a metaphorical 
has two major advantages. Firstly, it 
ing. Secondly, it allows higher 
to the "cocktail party* effect. This 
converse comfortably with one another in 
background noise, provided that they 
tfieir attention on a particular speaker, 
provide this identification, 
i number of manners. In a simple 
between the left and right channels, for 

volume on the left channel and none at 
full volume on the right channel and 
fjtayed at equal volume in both channels. 
, for example, introduce a delay to the 
channels, simulating the additional time 
listener's two ears in a regular physical 
khown to those skilled in the art. 

shows a Web page with Participant 
in the middle and Participant D's caller- 
displays the best identification 

This identification may include a 
>• information could be provided 
participant such as his address, 
click on "ear" and "eye" icons for each 
or disable their ability to hear her or 
A can also drag on a "mouth* icon to set 
and drag participants left and right to 
volume level of participants could be 
display - the further a participant's icon 
lower their volume level. Another 
speaks, their icon becomes * 



someone \ 
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Similarly, Participant B hears ParticiDants A, D and C in a metaphorical stereo 
spectrum from left to right. Because he finds the sound of participant A's voice close 
to that of Participant C, he has chosen to soparate them spatially as much as 
possible. This is done with the same type of GUI described with respect to 
Participant A above. 

Participant C hears a conventional f|>ur-way conference call, with the voices ~ 
of the three other Participants companded «md mixed together. As a result, she has 



difficulty distinguishing Participants B and D 
the call to some extent with the preference 



However, she has the flexibility to tailor 
or single voice dominating, adding noise 



filters, or other functionality via her proxy. This addition of functionality directly to 
PSTN customers is very significant. As explained in the background, PSTN services 
are driven by a supply model that only provides commodity services, and takes a 
long time to provide those limited services. There is a vast PSTN infrastructure which 
provides single monophonic lines into millions of homes and businesses, all of which 
are shackled with these limitations. The usv, of proxies in the manner of the invention 
provides greater flexibility and access to new services which may be implemented 
quickly and at very low cost. More details a ■© provided hereinafter regaining the 
preferred use of proxies. 

Participants who do not have the capability of interacting with the active 
network will have generic proxies assigned to them which are dictated by the nature 
of their telecommunication connections. Foi • < 

Participant only has PSTN access if that is t le connection the call manager has 
identified as the best connection during call setup. 

Participant D has a similar stereo arrangement, over which he has defined 
Participant A to reside on the left speaker te < 
speaker telephone and Participant C on both < 



creates a metaphorical stereo spectrum. Ot ler means are known in the art for 
carrying stereo over the PSTN, but such me hods generally require more complicated 
hardware at the Participant's end. Participant D has the same GUI as Participants A 
and B so he is able to control his proxy and mixer on the active network directly. 

As an example, exemplary signal processing software for participant A is 
presented as a block diagram in Figure 3. Voice streams from other participants 
arrive in different forms and need to be convBrted to a consistent form, companded 
and then mixed. They also need to be trans nitted across the radio link in an efficient 



ephone, Participant D on the right 
channels. This arrangement also 
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available, such as codecs and mixers, 
other listeners, and it takes slightly 
ate to their respective setups. 



form, and other signal processing such as echo cancellation/suppression may need 
to be done on the voice data. In this case, RealAudio has been chosen as the 
consistent form though MP3 or a number of other forms cpuld be used. RealAudio is 
particularly convenient as it is a realtime streaning standard that is well known in the 
5 industry and for which many tools are currently 
This type of processing is also required for the 
different forms for each participant which corre ; 

Not shown in Figure 3 are such function as encryption, tone controls and 
level control, though their implementation follows logically from the description 

10 provided herein. 

Specifically, pulse code modulated (PCM) voice streams are received from 
Participants C and D, which are connected to FiealAudio converters 64 and 66. PCM 
is the standard transmission form for audio in the PSTN. Since the voice signal 
received from Participant B is already in RealAudio format, which comprises data 

1 5 packets and is easily transported over IP, it is r ot necessary to convert it before 
passing its signal to the RealAudio mixer 68. 

The RealAudio mixer 68 combines the i looming audio signals in accordance 
with the participant's requirements. In Figure Ji, a bi-directional Activity and Control 
line is shown which interfaces the RealAudio mixer 68 with the PC 26 via an Ethernet 

20 card 72. The audio output of the RealAudio mi cer 68 goes to the Ethernet card 72 as 
well, and also to a PCM converter 74. 

This PCM converter 74 feeds the echo oancellor 76 with an audio signal that 
more or less matches the output from the partic apant's speaker 34. This way, the 
echo cancellor 76 can remove the speaker output signal that is inadvertently picked 

25 up by the microphone 36. The PCM signal leaving the echo cancellor 76 is 
converted to RealAudio at the voice coder 78. 

RealAudio packets are numbered sequentially to ensure that they are 
arranged in the proper order when they are decoded. Generally, it is not necessary 
to time stamp packets as the time delays are short, and the varied delays in data 

30 packets that result from their transport from different sources, or by different routes, 
will not generally be detectable by the participa nts. In fact, RealAudio may 
deliberately add a delay to the incoming signals by storing them in a buffer to absorb 
signal jitter. As the data is arriving in finite and 
inevitably be some degree of jitter, so buffering i 



distinct data packets, there will 

is preferred. A buffer that causes 20 
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to I 



PCM 



parti ciDants 



i wit i 



Internet 



mSec - 50 mSec delay is sufficient time to e 
and is not long enough to annoy the users 

Also, note that all audio signals 
Manager are in RealAudio form, and not 
5 digital interconnection more convenient. 

Other preferred aspects of the 

1. Call Setup 
The most important aspect of call 
where they can be found and then o 

1 0 preferred embodiment, the teleconfe 

the participants who are GUI-enable^ 
network of the identities of the 
network will make the connection 
Some of the participants will have 

15 telephone numbers. In each case, 

participant and establish the best 
Those participants without Internet 
reflect the resources they have 
software identifies a PSTN telephone 

20 assign a PSTN proxy to that 

In the preferred embodiment, all 
participants to the call, but for high 
participants should be controlled by z 

2. Telecommunications Operating 
25 The telecommunications operating 

unified control and access to all 
the functionality in and implied by 
of signal processing and control 
the commands of the callers. 
30 This contrasts with the "pure Internet 

cooperating tasks in all of the various 
of the processing through an 
calls, with no single program having 
makes it very difficult to optimize and 



bsorb the effects of jitter most of the time, 
a great extent, 
passjlng between the NetPort and NetPort 
This makes the transport over the 



tte 



possible 



i access 



:participent 
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teleconference are outlined as follows: 

sefup is the identification of the participants, 
course, creating the connections. In the 
'ence will be created by one or more of 
These participants will advise the 

and the call setup software on the 
the participants. 

addresses, while others will have 
call setup software will investigate the 
connection that it is aware of. 
will be assigned proxies which 
to. For example, if the call setup 
number as the best connection, it will 
unless advised otherwise. 

who are GUI-enabled can add 
teleconferences, addition of 
single participant 



ai;cessi 



partcipants 



security 1 



System 



system aspect of the invention provides 
system resources and networking links, with 

re 3. This represents a large collection 
functions connected together in response to 



ProtocoP approach which require 
computers to arrange to do their parts 
application-specific protocol built on socket() 
overview of the whole setup. This * 
manage the system, and each such 



sin 
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in 



application has to reinvent call process 
part of the underlying implementation ii 
invention is built as a middleware layer 
In telephony classic" one would not 
5 generality in software, but would make 

hardware that assumes all inputs and 
numbers to call in order to connect to it 
makes this a "closed system 0 , in which 
limitations on who can develop new 

10 3. Proxy 

It is preferred that the invention be 
graph model as described in the 
Patent Cooperation Treaty, Serial No 
for Telecommunications". While a 

1 5 and manage the entire connection, 

well as voice streams, that application i 
"agents" from "proxies". 
A proxy is a piece of software that acts 
connection. In this case Participants 

20 and Internet providers are separate 

contains data that represents the 
whether Participant A is already on the 
network's 28 voice trunk is getting full, 
do specific tasks, such as responding to 

25 managing a voice call in progress 

The terms proxy and agent are 
For the purposes of this document, the) 
agents, each of which handles a specia 
not comprise an immense block of code 

30 in its simplest form, is merely a 

as required, discarding them when their 
These agents are sent to parts of the 
going on and are connected to the signal 
through* a "controlling application". This 



ng. The invention uses socket() as 
a preferred embodiment, so that the 
on top of IP. 
attempt to set up something with this 
special "stereo conferencing server" 
cjutputs are PCM and would add speciaf 
The need for specialized hardware 
Innovation is slowed down by 
telecommunications applications 

appl ed to a network which employs a 
co-perjding patent application under the 
titled "Connection Manager 



t sometimes 



* 
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single application program is used to see 
including IP and videoCam connections as 
constructed by collecting together 

on behalf of a specific party to a 
A (through D, and each of the networks 
with separate proxies. A proxy 
preferjences and state of the party, such as 
elephone and whether the first active 
has components that are agents to 
off-hook on a telephone and 



2 nd l 



used interchangeably in the art. 
are distinct: a proxy is built out of 
situation. Therefore, the proxy does 
with all conceivable functionality, but 
or which instantiates software agents 
tasks have been completed, 
system in which signal processing is 
processing code or hardware 
architecture of proxies, agents and 
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the communication. In the event of 
around the failure, allowing the com 
transactional interaction techniques 
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controlling applications is what allows connection management applications to 
appreciate the whole structure of a connection while still being "owned" by 
several different parties. 

Proxies should persist in the preserice pf component failures, so that, for 
example, a user's forwarding instrui toons do not get lost during a crash. It is 
preferred that persistence be provic ed via a distributed database which is 
continuously updated, so that all concerned parties are aware of the status of 

a failure, the system is able to work 
munication to continue. Such 
are known in the art. 
In the "pure Internet Protocol" approach there is only custom software running 
on hardware belonging to the various parties involved and communicating 
through socket() mechanisms in an ad hoc protocol. The invention builds an 
additional structure on top of this 

In "telephony classic" there is a sindjle very large program that looks at a 
database for all users and decides what they would want to do. This program 
is too large to modify quickly, and can only be modified by the equipment 
manufacturer. Again, this approach is not flexible enough for rapid evolution 
of new features. The architecture cf the invention makes it easier to 
understand and modify software, without the same complexity, allowing the 
system to be open to software development, so that new features may be 
brought to market very quickly. 
Graph 

It is preferred that the invention be ipplied to a network which employs a 



graph model as described in the co 



Patent Cooperation Treaty, Serial No, 



pending patent application under the 



titled "Method and System for 



Configuring Communications Systems". Briefly, the graph model constructs 
the signal processing and communi sations structure, as a mathematical graph, 
which is later implemented by taking "filters" that implement the nodes out of 
libraries and modifying them, either by a dynamic linking process or by setting 
the IP addresses to which they make socket connections, to have the 
interconnection structure specified by the edges. This graph is also used for 
communication among the agents, as the data structure that defines a 
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cpnnection. An API layer that describe s characteristics of these graphs is 

added above the raw graph structure to assist in writing agents. 

In addition to filters, it is also preferred that this graph data packet contain 

calls to proxy agents required to set up the call. Proxies may also send their 

agents to collaborate on building and managing graphs. 

An application programming interface (API) converts a series of comparatively 



lower level instructions necessary to 



simple and high level functions into the I 

execute those functions, simplifying us 3 of an operating system. Using 
Windows APIs, for example, a program can open windows, files, and 
message boxes, as well as perform me re complicated tasks, by executing 
single instructions. 

The particulars of how an API for the ir vention is implemented are not critical, 
but it is desirable that a standard API be employed that expresses control, 



connection and negotiation processes, 
standard API simplifies the creation of 



a very logical and understandable form 
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including payment. The use of a 
lew features by third parties. 



A GUI is particularly well suited to the use of a graph model, as the GUI may 
present the assembled filters as define i by a graph data packet, to the user in 



It is also preferred that the GUI have 



the functionality to let the user modify t le graph data packet simply by altering 
filters and their interconnections. 
In the "pure Internet Protocol" approach the overall communications structure 
is not visible at all, while in the "telephony classic 0 approach it is possible for 
switch software to connect physical ports together, but not to pull functions out 
of libraries. The decision about what p Drts to connect together is explicitly 
made by the users by dialling telephone numbers in the "telephony classic" 
approach. 

Real Time Operating System (RTOS) 

Voice teleconferencing is a real time pr Dcedure, so RTOS should be used as 
known in the art. Generally, RTOS's divide code to be executed into smaller 
units of threads and functions, and then schedule the execution of these 
threads and functions to be performed Drior to specific deadlines. 
Distributed Operating System 

A distributed operating system is one in which portions of the software can run 
on different nodes. In the case of a telecommunications system, distribution 
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of software makes it easier to maint ain real time operation as there are more 
options available to schedule timely execution. As well, distributed operation 
improves scalability and speed. The* use of agents and proxies lends itself to 
the efficient use of a distributed system, in that agents and proxies may be 
assigned to run on different nodes cf the system. Ideally, agents will be 
located close to where they are reqi lired, to minimize time delays in 
communicating with the entities they represent. Such a suitable distributed 
RTOS is described in the co-pendin j patent application under the Patent 

Cooperation Treaty, Serial No. . titled "Distributed Telco\ 

Figure 4 presents a block diagram of an exemplary operating system 
architecture in a preferred embodimsnt of the invention. A distributed 
communications substrate 80 is interposed between user processes and the 
underlying machines, so that processes can generally be moved from one 
machine to another without being av/are of it, either to distribute load or to 
recover from failures. 

Processes running in the system come from different sources and accordingly 
get different treatment in terms of ths trade-off between security and 
performance. Call-processing functions acting on behalf of the end users run 
in a protected "sandbox" environment on a virtual machine. Those working 
on behalf of the network provider may run there, but may also be 
implemented directly as processes running on the network operating system. 
User processes running as "filters", with the hard real-time demands that 
come from being in the signal path, and also run directly on the 
communications substrate 80. Processes belonging to different users are 
protected from each other by the us jal operating system mechanisms such 
as memory mapping and file privileges, but the source is also reviewed by the 
network administrator. Filter processes on the same machine and part of the 
same call may share an address space and a thread of control, with data 
being passed with a function call mechanism and with connections to other 
hardware being handled by a stub ttiat adapts a function call to a socket-type 
mechanism. These filters would still be dynamically linked, even with the 
function-call mechanism. 

Signals pass through filter processed F, which also implement drivers and 
performance-sensitive functions on behalf of the network. Call processing on 
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allocated to a physical network: each 
nodes, for example, and sometimes 
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behalf of users is handled by CP pre cesses running in a secure virtual 
machine VM environment, which also includes checkpointing functions that 
can transfer control on failure to a "£ host machine" 82. All these processes 
run on the common software communications layer 80, which places them on 
appropriate physical systems and arranges for their connections. Server 
processes S also run on the commu locations substrate 80, but do not have ~ 
the hard real-time constraints of the filter processes. Secure call-processing 
functions are one type of server process. 
Mapper 

The allocation of tasks and network fcapacity to different communications 
graphs is done by an optimizer calle J the mapper. There are in general 
many ways that a graph representing a desired communication can be 



of the filters can run on several different 
i (there are several types of links over 
which data can be carried. The simplest embodiment uses hints from the 
proxies about where to put radio link s, that is, after the voice coder, and then 
applies a "greedy algorithm 0 to put computing resources as close to the net- 
work edge as possible. A good map :>er should be a distributed application in 

ocal knowledge to the greatest extent 
possible. It is not essential to get a £ lobal optimum, as long as resources are 
not seriously wasted. 

In the case of the invention, the most significant resource management 
problem is the handling of the voice sitreams. In having a separate mixer for 
each of N participants, each mixer w II receive (N-1) voice streams. The 
mapper must balance the benefit of distributing the mixers among various 



processors against the extra cost of 



is quality of service (QoS). Methods 



each set of participants and network 
6. Negotiation 

It is preferred that the architecture for 



system described in the co-pending patent application under the Patent 



ransporting redundant audio signals. 



The factor that governs these decisic n in the implementation of the invention 



of distributing such real time loads are 



known in the art, but in the case of the invention, the solution will vary with 



opography. 



agents provide for use of the negotiation 
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Cooperation Treaty, Serial No., 



Negotiating Telecommunication Re sources 



Many users are competing at once 
(including its computing capacity) 



frequent. With differential service s 
defined, but the definition needs to 
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titled "Method and System for 



> for the shared resources of the network 
t is preferred to apply a market model to 
resolve this contention: agents for tie parties involved offer and demand 
payment as part of connection setu :>, and a connection does hot happen until ~ 
all parties have accepted it. A caller can choose to try a connection at a 
reduced quality level if the cost of the high quality connection becomes too 
high. For example, on Christmas Day, the load may be temporarily high, so 
users can expect to get through wit i reduced voice quality rather than getting 
a busy signal. 

In "pure Internet Protocol", temporary congestion is resolved on a "best 
efforts" basis and packets may be t ilmost arbitrarily thrown out, and at a 
longer time-scale by overprovisioning the network so that failures are not too 

small number of priority classes are 
ae managed. The market model of the 
invention can be used to manage differential service, allocating high priority 
access in such a way as to permit c uarantees on service. 
In "telephony classic" contention is managed by call admission (first-come, 
first served) and again the network is overprovisioned so that failures are not 
too frequent 

Negotiation management may be ir iplemented by having a negotiation agent 
for each of the user terminals and for each of the multiple telecommunications 
networks. Each negotiation agents is operable to execute somewhere on the 
telecommunications system, for example, on the active network, and 
represents the interests of its respective party in negotiating communication 
over the telecommunications network. This is done by identify participants in 
the negotiation and then passing a graph data packet which describe the 
proposed connection, to each participant for their consideration. Each 
negotiation agent may either accept, reject or revise it to make a new 
proposal to the other negotiation agents. 

When all or part of the graph data packet is to be executed, a device simply 
assembles the listed filters in the manner defined in the graph data packet. 
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implemented with a strong security 



It is also preferred that the invention be i 
mechanism that protects proxies from erroneo js or malicious code in other proxies. 
As well, it is presently desirable that proxies ar d agents be written in Java™, but 
another language with similar advantages could also be used. Advantages of Java™ 
5 include: 

a. excellent security 

b. a large community of experienced developers 

c. object oriented code structure 

d. simple net-based distribution mechanism 
10 A telecommunications system impleme ited with the functionality described 

above provides a foundation for the mixed media applications of the future, and for 
greater flexibility and power to existing services such as high bandwidth telephone, 
and Internet gaming. 

Other options for implementation of the invention include: 
15 1. Companding 

Companding techniques use "compression" algorithms that try to adjust gains 
(smoothly) so as to keep a signal's level more constant and "expansion" 
algorithms that adjust gains to exaggerate signal-level variations. Some 
techniques used in audio are frequency - 
20 companding which adjusts filter cutoffs 

signal levels are low. 

An extreme example of expansion is M s luelch" in which signals with power 
level below a certain threshold are turned off completely to minimize idling 
noise. In telephony the most common 1 

25 opposed to "cancellation", in which the 

its gain reduced, which reduces the loo 3 gain for echoing and feedback 
oscillation. Companders use around 5-50 operations per sample. 
Instantaneous companders work on a £ ample-by-sample basis, and the 
common A-law case is covered under " ;oders" below, 

30 2. Voice coding 

Voice coders are used to reduce the bandwidth requirements for voice 
signals. There are many types, but broadly they can act on the waveform, 
minimizing some mathematical measurs like error power; they can model the 
source; or they can model what the ear will notice. Coding for compression is 



dependent, such as Dolby 
to suppress background hiss when 



variant is "echo suppression", as 
signal path from the quieter user has 
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an active research area, and a steajdy stream of new coders is likely to 
appear. 



telephony classic" uses waveform 
law). Sampling is done at 8kHz on 



coding in the form of 8kHz A-law (or p- 
a signal filtered to pass the range from 



on speech quality and intelligibility, 
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300Hz to 3300Hz. The passband was defined to get good subjective scores 



and the sample rate is designed with a 



33% margin over the Nyquist minirr urn in a trade-off between network and 
prefilter costs. A-law and p-law are specialized 8-bit floating-point 
representations, chosen as a way t> get roughly constant signal-to-noise over 
a wide range of signal levels. By comparison, compact disc (CD) sound is 

44.1kHz, which requires roughly 24 times 
the bandwidth and the use of a T1 tine. Because speech varies slowly from 
sample to sample, the same quality can be had for roughly half the bandwidth 
with ADPCM (adaptive differential pulse-code modulation) which, roughly 
speaking, digitizes the derivative in stead. 

Most digital cell-phones use a variant of linear prediction coding, which tries to 

of a sound source that simulates the vocal 
cords or airflow and which in turn d ives a filter that models the larynx. This 
requires less bandwidth than waveiorm coding because the larynx moves 
more slowly than the waveform, bu : works badly for anything other than 
speech or even for speech in a noisy environment. These "source coders" 

I currently produce tolerable speech at 
output rates anywhere from 4kb/s up. Atypical modern coder uses about 
SOMIPs of DSP capacity. Coders typically operate on 20msec frames of data, 
and hence add at least that much c elay to the signal path. 
Source coders typically try to detect silence, and avoiding the transmission of 
silence typically saves about 50% of bandwidth on average. At the decoding 
side it is conventional to replace silence with "comfort noise" so that the 

live. 

' music, because it would be necessary to 



listeners know the connection is sti 
Source coding is difficult to use for 



model a large number of different instruments alone and in combination, so 
early digital audio such as CD and DAT, just used waveform coding with 
enough bandwidth and dynamic range to satisfy (more or less) the human 
ear. Minidisc and digital compact cassettes brought in coding that reduced 
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CD bandwidth by a factor of about 10 b/ using psychoacoustics. 



Psychoacoustics applies, in particular, 
mask nearby ones for normal ears, anc 
transmitting the inaudible components 
rate-adapted, as in RealAudio, and is a 



(nasking effects, where loud tones 
bandwidth can be saved by not 
This type of technique can also be 
good candidate for high-quality 



speech applications in the system of tho invention. 



Conventional filters, companders and s 



200 kpixels / frame * 3 colours * 8 bits / 
what 3G wireless is built to handle, but 
hence the 3G requirement for that rate. 



capacity when the image changes sudc enly. 
At the low-quality end, videoconferencing is usually done at 128kh/s. At this 
rate the coding process adds hundreds of msec of delay and the picture is 
poor. 

If there is high demand for full-motion video, then 5MHz slots will not have 
sufficient capacity, but 20MHz slots ancji < 
could support 10-40 users at that rate. 
The network operating system could initiate processes in the end-user's PC 
so that video services can be set up easily. 
Other applications 
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milar components will not work on a 



coded signal, so it is standard to decompress before filtering. In some cases 
this may be avoided, for example, N-ws y combining can take advantage of 
silence to do companding at no additional cost of bandwidth, and only needs 
to decode and recede during bi-directio tal conversation. 
MPEG (Motion Picture Experts Grou|>) 

MPEG coders do the same type of tiling for video signals that perceptual 
coders do for music. Components of a 

frequencies are digitized at low resolution, using 8*8 discrete cosine 
transforms to do the filtering, and using "motion estimation" so that 
components of an image that can be dorived from adjacent frames are not 
retransmitted. MPEG decoding is pref€ rably left for the end-user's PC, 
because it is very demanding and beca jse specialized hardware exists for it. 
However, the traffic properties are an inportant consideration in implementing 
the invention. Straight digitized television requires roughly 30 frames / sec 1 



colour, for 144Mb/s. That is beyond 
MPEG2 gives similar quality at 2Mb/s; 
MPEG2 is also bursty, needing more 



generous use of antenna diversity 
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Other applications such as animated 
locations, voice activation, automat 
signal shaping to compensate for 
or software in the system, are all 
5 invention. 

Examples have been shown to 
but the number of variations is by no mearjs 
could be made for any telephony device, 
machines, pagers, point of sale computers!, 
10 private branch exchanges. While particular 
have been shown and described, it is clear 
made to such embodiments without depar 
invention. 

The invention could also be 
1 5 Internet and PSTN networks. For exampty 
the functionality of the invention similar to 
PSTN a specialized server could be attached 
implementations would not have all the 
certain aspects of its teachings. 
20 The method steps of the invention 

machine code stored in a variety of formatjs 
Such code is described generically herein 
program for simplification. Clearly, the 
with the code of other programs, implemented 
25 calls or by other techniques as known in 
The embodiments of the invention 
or similar device programmed in the manner 
an electronic system which is provided wit 
Similarly, an electronic memory means su 
30 Access Memory (RAM), Read Only Memo(ry 
storage media known in the art, may be 
As well, electronic signals representing th£se 
via a communication network. 



video, stereo input at the participant's 
c gain control (AGC) at the user's PC, and 
frequency response of certain devices 
Mown in the art, and easily applied to the 



ithie 
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demonstrate various aspects of the invention, " 
complete. Comparable implementations 
including personal digital assistants, fax 
amateur radios, local area networks or 
embodiments of the present invention 
that changes and modifications may be 
ing from the true scope and spirit of the 

implemented to a lesser extent on existing 

, Internet servers could be given much of 
applications such as NetMeeting. On the 
to a class 5 switch. These 
behefits of the invention, but could apply 

nay be embodied in sets of executable 
such as object code or source code, 
as programming code, or a computer 
executable machine code may be integrated 
as subroutines, by external program 
art. 

may be executed by a computer processor 
of method steps, or may be executed by 
i means for executing these steps, 
h computer diskettes, CD-Roms, Random 
(ROM) or similar computer software 
programmed to execute such method steps, 
method steps may also be transmitted 



r 
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It would also be clear to one skilled in the art that this invention need not be 
limited to the described scope of computers and computer systems. The principles of 
the invention could be applied to citizen's band radio, amateur radio, or packet radio. 
Again, such implementations would be dear to one skilled in the art, and do not take 
5 away from the invention. 
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WHAT IS CLAIMED IS: 

1 . A system for teleconferencing comprising 
three or more user terminals, each having 
a telecommunications network interconnecting 

transport data to and from said use 
separate modular mixing software for each 
said telecommunications network, 
to receive separate audio signals 

user terminals; and 
to combine said separate audio signals 
said respective user termin; 
respective user terminal. 



an audio input and an audio output; 

said user terminals and operable to 
* terminals; 

respective user terminal,, executing on 
operable: 

said audio outputs of the others of said 



end 
from 



aid 



each 



2. A system as claimed in claim 1 , further 
modular connection management software 
said three or more user terminals 
including a connection proxy for 
said telecommunications network; 
each of said connection proxies executing 
to represent its owner's interests in 
recognizing the limitations o 



3. A system as claimed in claim 2, further 
a mapper for locating said separate modular 
terminal for execution on different 
network, trading off delay time in 
computational power available in 



A system as claimed in claim 3, wherein 
comprises multiple telecommunications 
and protocols, each of said multiple 
own connection proxy. 
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into a signal for said audio input of 
I which correlates to the needs of said 



composing: 
for establishing interconnections between 
said separate modular mixing software, 
of said three or more user terminals and 



cind 

on said system and being operable: 
managing the teleconference by 
its resources. 



comprising: 
mixing software for each respective user 
rbuters of said telecommunications 
communicating data between routers with 
to maintain quality of service. 



order 1 



said telecommunications network 
networks with varied transport media 
telecommunications networks having its 
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said three or more user terminals in 
said telecommunications network; 



5. A system as claimed in claim 4, further somprising: 
negotiation management software including a negotiation agent for each of said user 

terminals and said multiple telecommunications networks, each of said 
negotiation agents being operable: 
to execute on said system; and 
to representing the interests of each of 

negotiating communication over i 
said negotiation management software being o arable: 

to identify negotiation agents participating in a negotiation; 
to implement a negotiation discipline wMch allows each said participating 

negotiation agent to consider a communication contract and either 

accept or revise said communication contract; and 
to respond to said negotiation being successful by executing said 

communication contract. 

6. A system as claimed in claim 5, whereiiji said separate modular mixing 
software is operable to combine said separate signals into two or more audio 
channels which define a metaphorical physical space, each user terminal 
having a simulated position within said metaphorical physical space whereby 
individual users may be recognized by their particular position in said space. 
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A system as claimed in claim 6, wherein 
from said respective user to emphasize 
the corresponding audio signal prior to 



combining. 



A system as claimed in claim 7, whereiji 
comprises a personal computer having 
speakers, and said respective mixer software 
separate signals into two audio channe 



said mixer is responsive to a request 
a particular user's voice by amplifying 



at least one of said user terminals 
a stereo sound card and stereophonic 
is operable to combine said 

s, ' 
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A system as claimed in claim 8, wherein 
comprises a connection to a PSTN 
lines, and said respective mixer software 
separate signals into two audio channels 



10. 



A system as claimed in claim 9, wherein each said connection proxy 
comprises: 
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at least one-of said user terminals 
network via two monophonic telephone 
is operable to combine said 



multiple software agents each being opera Die to perform a specific task; and 
a proxy object operable to instantiate particular ones of said multiple software agents 

in response to requirements of communications made over said 

telecommunications system. 



mixing 



11. A server for teleconferencing comprising 
means for interconnecting user terminals 
terminals; 

means for executing separate modular mining 
terminal, said separate modular 
means for receiving separate audio 
others of said user terminal:; 
means for combining said separate 
input of said respective use 
said respective user terminal 



snd transporting data to and from said user 



12. A method of teleconferencing comprising 
receiving, at a separate modular mixer repjresenti 
more user terminals and executing 
separate audio signals from audio 
and 

combining said separate audio signals into 

respective user terminal which correlates 
terminal. 



software for each respective user 
software including: 
signals from said audio outputs of the 
and 

audio signals into a signal for said audio 
terminal which correlates to the needs of 



the steps of: 

ing a respective one of three or 
on a telecommunications network, 
Outputs of the others of said user terminals; 

a signal for an audio input of said 

to the needs of said respective user 
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13. A computer data signal embodied in a carrier wave, said computer data signal 
comprising a set of machine executabl^ code being executable by a computer 
to perform the steps of claim 12. 
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14. 



A computer readable storage medium storing a set of machine executable 
code, said set of machine executable cade being executable by a computer - 
server to perform the steps of claim 12. 
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